Refactor codegen backends in bootstrap #144787

Kobzol · 2025-08-01T16:00:14Z

This PR refactors the codegen backend steps, in preparation to make more progress on the integration of the GCC codegen backend in bootstrap. It does several things:

Splits the CodegenBackend step into two, one for clif and another one for gcc. Even though their code is mostly similar, that's IMO mostly fake similarity, and they do (or will) ultimately require different handling. This was already visible in the requirement of building GCC for cg_gcc, of course.
It is now possible to build both backends (and dist cranelift) even if they are not specified in rust.codegen-backends. It was quite weird that it wasn't possible to even invoke the corresponding codegen backend if the backend wasn't specified in that array, as that array should ideally only change defaults (see later below).
Changes the path specification of these steps to an alias. In other words, instead of compiler/rustc_codegen_cranelift, the step is now built only using rustc_codegen_cranelift or cg_clif. This is done to avoid an annoying clash with x build compiler, which would otherwise build both codegen backends after the 2) change.
Made the copying of codegen backend artifacts more explicit, in particular in the Assemble step.
Codifies the semantics of rust.codegen-backends, which now only affects the defaults of whether a codegen backend will be included in rustc's sysroot and whether it will be disted in x dist by default. We can change the behavior later, e.g. to dist cranelift by default in x dist once it becomes stabilized. Currently I left the existing behavior that we use on CI, I just tried to document it better.

I don't think that this requires a change tracker entry, because the defaults should work the same as before. It is just now possible to do x build/dist rustc_codegen_cranelift even if CLIF is not in the codegen-backends array. It is no longer possible to do ./x build compiler/rustc_codegen_cranelift though, not sure if that requires a change tracker entry.

There is one thing that I didn't touch yet, and that is the fact that rust.codegen-backends not only affects the default behavior of x dist w.r.t. Cranelift, but also of x test. In other words, x test rustc_codegen_cranelift still does not hing if cranelift isn't in rust.codegen-backends. I plan to take a look at this once I get to refactoring the test steps.

r? @jieyouxu

rustbot · 2025-08-01T16:00:19Z

jieyouxu is currently at their maximum review capacity.
They may take a while to respond.

rustbot · 2025-08-01T16:00:22Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR modifies bootstrap.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

bootstrap.example.toml

jieyouxu · 2025-08-05T11:52:35Z

@rustbot author

Kobzol · 2025-08-05T14:50:00Z

@rustbot ready

jieyouxu

Thanks, I have some nits/questions

src/bootstrap/src/core/build_steps/compile.rs

src/bootstrap/src/core/build_steps/dist.rs

src/bootstrap/src/core/builder/tests.rs

bootstrap.example.toml

Kobzol · 2025-08-06T12:26:03Z

Pushed review changes.

jieyouxu

Thanks, two nits because I only just noticed

bootstrap.example.toml

src/bootstrap/src/core/build_steps/compile.rs

src/bootstrap/src/core/build_steps/dist.rs

src/bootstrap/src/core/builder/tests.rs

jieyouxu

Thanks

jieyouxu · 2025-08-06T13:44:21Z

@bors r+ rollup=never (build system changes, just in case)

bors · 2025-08-06T13:44:24Z

📌 Commit e89c64f has been approved by jieyouxu

It is now in the queue for this repository.

Zalathar · 2025-08-07T07:38:01Z

Let's run a never PR while I investigate the recent rollup failure.

@bors p=1

Zalathar · 2025-08-07T08:05:53Z

Undoing my previous priority bump, since the immediate scheduling concern was solved.

@bors p=0

Kobzol · 2025-08-08T09:38:00Z

Rebased and blessed snapshot tests.

@bors r=jieyouxu

bors · 2025-08-08T09:38:03Z

📌 Commit 1a2ee80 has been approved by jieyouxu

It is now in the queue for this repository.

bors · 2025-08-08T15:24:24Z

⌛ Testing commit 1a2ee80 with merge e723c3c...

Refactor codegen backends in bootstrap This PR refactors the codegen backend steps, in preparation to make more progress on the integration of the GCC codegen backend in bootstrap. It does several things: 1) Splits the `CodegenBackend` step into two, one for clif and another one for gcc. Even though their code is mostly similar, that's IMO mostly fake similarity, and they do (or will) ultimately require different handling. This was already visible in the requirement of building GCC for cg_gcc, of course. 2) It is now possible to build both backends (and dist cranelift) even if they are not specified in `rust.codegen-backends`. It was quite weird that it wasn't possible to even invoke the corresponding codegen backend if the backend wasn't specified in that array, as that array should ideally only change defaults (see later below). 3) Changes the path specification of these steps to an alias. In other words, instead of `compiler/rustc_codegen_cranelift`, the step is now built only using `rustc_codegen_cranelift` or `cg_clif`. This is done to avoid an annoying clash with `x build compiler`, which would otherwise build both codegen backends after the 2) change. 4) Made the copying of codegen backend artifacts more explicit, in particular in the `Assemble` step. 5) Codifies the semantics of `rust.codegen-backends`, which now only affects the defaults of whether a codegen backend will be included in rustc's sysroot and whether it will be disted in `x dist` by default. We can change the behavior later, e.g. to dist cranelift by default in `x dist` once it becomes stabilized. Currently I left the existing behavior that we use on CI, I just tried to document it better. I don't think that this requires a change tracker entry, because the defaults should work the same as before. It is just now possible to do `x build/dist rustc_codegen_cranelift` even if CLIF is not in the `codegen-backends` array. It is no longer possible to do ` ./x build compiler/rustc_codegen_cranelift` though, not sure if that requires a change tracker entry. There is one thing that I didn't touch yet, and that is the fact that `rust.codegen-backends` not only affects the default behavior of `x dist` w.r.t. Cranelift, but also of `x test`. In other words, `x test rustc_codegen_cranelift` still does not hing if cranelift isn't in `rust.codegen-backends`. I plan to take a look at this once I get to refactoring the test steps. r? `@jieyouxu`

rust-log-analyzer · 2025-08-08T17:07:39Z

The job x86_64-gnu-aux failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

failures:

---- f32::test_gamma stdout ----

thread 'f32::test_gamma' (194438) panicked at library/std/tests/floats/f32.rs:198:5:
2.0000012 is not approximately equal to 2.0 (threshold 1e-6, difference 1.1920929e-6)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
note: in Miri, you may have to set `MIRIFLAGS=-Zmiri-env-forward=RUST_BACKTRACE` for the environment variable to have an effect


failures:
    f32::test_gamma

test result: FAILED. 28 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.81s

error: test failed, to rerun pass `-p std --test floats`
Build completed unsuccessfully in 0:04:39
make: *** [Makefile:58: check-aux] Error 1
  local time: Fri Aug  8 17:07:15 UTC 2025
  network time: Fri, 08 Aug 2025 17:07:15 GMT
##[error]Process completed with exit code 2.
Post job cleanup.

bors · 2025-08-08T17:08:03Z

💔 Test failed - checks-actions

lqd · 2025-08-08T17:08:30Z

I knew it, we may need to close the tree

Kobzol · 2025-08-08T17:12:10Z

@bors treeclosed=100 A test is failing on CI.

I guess we just comment out the test for now?

Revert #143906 This reverts commit 71f0469, reversing changes made to 995ca3e. Reverts #143906, which was merged in #145043. It seems like it is causing test failures on CI that block merges (#144787 (comment). try-job: x86_64-gnu-aux

Kobzol · 2025-08-08T20:52:25Z

#145116 landed, so reopening the tree.

@bors treeclosed-

Kobzol · 2025-08-08T20:53:02Z

@bors retry

rustbot assigned jieyouxu Aug 1, 2025

Kobzol changed the title ~~Refactor codegen backends~~ Refactor codegen backends in bootstrap Aug 1, 2025

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Aug 1, 2025

This comment has been minimized.

Sign in to view

GuillaumeGomez reviewed Aug 3, 2025

View reviewed changes

bootstrap.example.toml Show resolved Hide resolved

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 5, 2025

Kobzol force-pushed the codegen-backend-restructure branch from 588f96f to 0a3b2d7 Compare August 5, 2025 14:49

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 5, 2025

jieyouxu requested changes Aug 6, 2025

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 6, 2025

jieyouxu approved these changes Aug 6, 2025

View reviewed changes

bootstrap.example.toml Show resolved Hide resolved

src/bootstrap/src/core/build_steps/compile.rs Show resolved Hide resolved

src/bootstrap/src/core/build_steps/dist.rs Show resolved Hide resolved

src/bootstrap/src/core/builder/tests.rs Show resolved Hide resolved

Kobzol force-pushed the codegen-backend-restructure branch from 4235812 to e89c64f Compare August 6, 2025 13:35

jieyouxu approved these changes Aug 6, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 6, 2025

Shourya742 approved these changes Aug 7, 2025

View reviewed changes

Kobzol force-pushed the codegen-backend-restructure branch from e89c64f to 1a2ee80 Compare August 8, 2025 09:37

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 8, 2025

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 8, 2025

This was referenced Aug 8, 2025

add a scope for if let guard temporaries and bindings #143376

Open

Miri: non-deterministic floating point operations in foreign_items #143906

Merged

Revert #143906 #145116

Merged

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 8, 2025

Refactor codegen backends in bootstrap #144787

Are you sure you want to change the base?

Refactor codegen backends in bootstrap #144787

Conversation

Kobzol commented Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Aug 1, 2025

Uh oh!

rustbot commented Aug 1, 2025

Uh oh!

This comment has been minimized.

Uh oh!

jieyouxu commented Aug 5, 2025

Uh oh!

Kobzol commented Aug 5, 2025

Uh oh!

jieyouxu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Kobzol commented Aug 6, 2025

Uh oh!

jieyouxu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jieyouxu left a comment

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Aug 6, 2025

Uh oh!

bors commented Aug 6, 2025

Uh oh!

Zalathar commented Aug 7, 2025

Uh oh!

Zalathar commented Aug 7, 2025

Uh oh!

Kobzol commented Aug 8, 2025

Uh oh!

bors commented Aug 8, 2025

Uh oh!

bors commented Aug 8, 2025

Uh oh!

rust-log-analyzer commented Aug 8, 2025

Uh oh!

bors commented Aug 8, 2025

Uh oh!

lqd commented Aug 8, 2025

Uh oh!

Kobzol commented Aug 8, 2025

Uh oh!

Kobzol commented Aug 8, 2025

Uh oh!

Kobzol commented Aug 8, 2025

Uh oh!

Uh oh!

Kobzol commented Aug 1, 2025 •

edited

Loading